Max Pellert (https://mpellert.at)
Deep Learning for the Social Sciences
From now on, please find all class materials in this repo: https://github.com/DLSS-24/DLSS-24
Univariate regression problem (one output, real value)
Supervised learning model = mapping from one or more inputs to one or more outputs
Computing the inputs from the outputs = inference
Example:
Input is age and mileage of secondhand Toyota Prius
Output is estimated price of car
Model is a mathematical equation, but better and more generally: model is a family of equations
Model includes parameters
Parameters affect outcome of equations
Training a model = finding parameters that predict outputs “well” from inputs for a training dataset of input/output pairs
Check Appendix A of “Understanding Deep Learning”
| Input | Variables are always indicated with Roman letters Normal = scalar Bold = vector CAPITAL BOLD = matrix |
|
| Output |
| Model | Functions are always indicated with square brackets Normal = returns scalar Bold = returns vector CAPITAL BOLD = returns matrix |
| Input | Structured or tabular data | |
| Output | ||
| Model | ||
| Parameters | Parameters are always Greek letters | |
| Model |
We use a training dataset of I pairs of input/output examples:
Loss function or cost function measures how bad the model is at relating input to output for the examples:
Or short:
Loss function: returns a scalar that is smaller when model maps inputs to outputs better
During training, we try to find the parameters that minimize the loss:
To test the model, we evaluate it on a separate test dataset of input/output pairs
Crucially, it must not have seen that data during training (suspiciously high, almost perfect performance on the test set is an indicator that there may have been a spillover)
Testing allows us to see how well it generalizes to “new data”
Still, the test data is usually from the same domain and collected in the same way as the training data, so external validity can be low although test set performance is high
Always critically assess and try to assess performance “in the wild” to establish the model’s limits
This is clearly where your ways to think from the social sciences can come in very handy!
Model
Parameters
Loss function:
Least squares loss function
Loss function:
Least squares loss function
Loss function:
Least squares loss function
Loss function:
Least squares loss function